Engineering posts about Data Lineage
Curated summaries and key learnings for engineers working with Data Lineage.
Network Quality Is a Revenue Problem, Not a Technical One
The article emphasizes the importance of integrating network performance data with commercial data in telecommunications to enhance revenue management. It highlights the limitations of traditional...
Why Your OEE Dashboard Is Lying to You
The article highlights the discrepancies between KPI dashboards and the actual performance of manufacturing equipment, specifically focusing on Overall Equipment Effectiveness (OEE). It identifies...
Interoperability Between Unity Catalog and Google BigQuery via Catalog Federation
The article announces the interoperability between Databricks Unity Catalog and Google BigQuery through catalog federation, allowing users to access data from either platform without duplication....
Announcing the Public Preview of Lakeflow Designer
Lakeflow Designer is a no-code, AI-native tool designed for data preparation and analytics within the Databricks platform. It enables users, including analysts and domain experts, to prepare and...
Building with Databricks Document Intelligence and Lakeflow
The article discusses the challenges of accessing unstructured enterprise knowledge trapped in documents and presents Databricks Document Intelligence and Lakeflow as solutions for automating...
Why agentic analytics starts with a well-governed data layer
The article emphasizes the critical role of a well-governed data layer in enabling effective AI and analytics. It highlights the challenges posed by legacy business intelligence (BI) systems, which...
Inside Snap's Experimentation Platform: Leveraging NVIDIA GPUs for Accelerated Data Processing
The article details Snap's transition to a GPU-accelerated data processing pipeline using Apache Spark and NVIDIA's RAPIDS Accelerator. It outlines the challenges faced during the migration,...
Unified data discovery with business context in Unity Catalog
The article introduces the Databricks Discover experience, which aims to streamline data discovery by embedding business context directly into Unity Catalog. As organizations grapple with the...
Adaptive Data Governance for EU Regulatory Change
The article outlines the evolving landscape of data governance in response to new EU regulations such as the Digital Omnibus and DORA. It emphasizes the need for financial institutions to adopt...
Redefining impact as a data scientist
The article outlines how data science can redefine its impact in complex systems, particularly in billing infrastructures. It emphasizes that impactful data science work often transcends traditional...
How Databricks System Tables Help Data Engineers Achieve Advanced Observability
The article discusses how Databricks System Tables facilitate advanced observability for data engineers by providing queryable telemetry data related to jobs, pipelines, clusters, and billing. It...
Business Analytics: Essential Tools, Techniques and Skills for Data-Driven Success
The article provides a comprehensive overview of business analytics, emphasizing its role in data-driven decision-making within organizations. It categorizes analytics into four core types:...
Against the Clock: How Data 360 Launched the Informatica Help Agent in 24 Days
The article outlines the rapid development of the Informatica Help Agent, achieved in just 24 days through the innovative use of Data 360. The team focused on transforming 100,000 unstructured...
BCBS 239 Compliance in the Age of AI: Turning Regulatory Burden into Strategic Advantage
The article explores how financial institutions can leverage Databricks to automate compliance with BCBS 239, a regulatory standard for risk data aggregation and reporting. It highlights the...
Top 10 Questions You Asked About Databricks Clean Rooms, Answered
The article discusses Databricks Clean Rooms, a secure environment for collaborative analysis of sensitive data without exposing raw records. It outlines how organizations can utilize Clean Rooms to...